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DETAILED ACTION 

Response to Amendment 

1 . Claims 1-30 are pending, claims 1 and 12 are amended, and claims 25-30 are 
newly added. 

Response to Arguments 

2. Applicant's arguments with respect to claims 1 and 12 have been considered but 
are moot in view of the new ground(s) of rejection. 

3. Applicant's arguments, see page 8, filed 6/5/2008, with respect to the rejection(s) 
of claim(s) 23 and 24 under 35 U.S.C. § 112 have been fully considered and are 
persuasive. Therefore, the rejection has been withdrawn. However, upon further 
consideration, a new ground(s) of rejection is made in view of US 2001/0015759 
(Squibbs) in view of US 2001/0041020 (Shaffer). 

Claim Rejections - 35 USC § 103 

4. The text of those sections of Title 35, U.S. Code not included in this action can 
be found in a prior Office action. 

5. Claims 1-6, 8, 12-17 and 19 are rejected under 35 U.S.C. 103(a) as being 
unpatentable over US 6,469,698 (Fukahori) in view of US 2004/0021780 (Kogan). 

6. As to claim 1, Fukahori teaches a method of correlating an image with 
information associated with information associated with the image comprising: 

identifying image metadata for the image (step S101 and S102 in Fig. 2), wherein 
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the image metadata includes information associated with the conditions at the time of 
image capture (Column 3 lines 7-17); 

searching one or more information sources using parameters in the image 
metadata to collect inference information from the information sources, the inference 
information including location identification information (Column 4 lines 8-51); and 

displaying the image inference information (Fig. 4 and Column 4 lines 8-51). 

What Fukahori does not explicitly teach is that filed of view information is 
included in the inference information. However, Kogan teaches using location collecting 
location information and field of view information ([0016] and [0018]). Therefore, it would 
have been obvious to one of ordinary skill in the art at the time the invention was made 
to have used metadata for an image that includes information associated with the 
conditions at the time of capture to collect information including location identification 
information and field of view information, as is done in Kogan, as this would allow for 
objects in an image (monuments, geographical formations, etc.) to be identified for a 
user accurately. 

7. As to claim 2, see the rejection of claim 1 and note that Fukahori further teaches 
the method of claim 1 further comprising: 

receiving one or more inputs from the user identifying selected inference 
information (Column 4 line 52 to Column 5 line 6); and 

adding the selected inference information to an image file for the image (Column 
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5 line 49 to Column 6 line 13). 
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8. As to claim 3, see the rejection of claim 1 and note that Fukahori further teaches 
the method of claim 1 further comprising: 

receiving one or more inputs from the user identifying selected inference 
information (Column 4 line 52 to Column 5 line 6); and 

adding the selected inference information to an inference metadata file linked to 
the image (Column 5 line 49 to Column 6 line 13). 

9. As to claim 4, see the rejection of claim 1 and note the Fukahori further teaches 
the method of claim 1 wherein the image metadata includes parameters selected from 
the group consisting of: 

time of image capture; 
date of image capture; 

location of image capture (Column 3 lines 7-17); 

direction of image capture device during image capture; and 

angle of image capture device during image capture. 

10. As to claim 5, see the rejection of claim 1 and note that Fukahori further teaches 
the method of claim 1 wherein the image metadata includes a latitude and longitude of 
the image capture device (Column 3 lines 7-17). 
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11. As to claim 6, see the rejection of claim 1 and note that Fukahori further teaches 
the method of claim 1 wherein the image metadata includes location information 
generated by tracking multiple earth-orbiting satellites (Column 3 lines 7-17). 

12. As to claim 8, see the rejection of claim 1 and note that Fukahori further teaches 
the method of claim 1 wherein the inference information is selected from the group 
consisting of: 

landmarks located near the image; 
weather at the time of image capture; 

information related to the location where the image was captured (Column 4 lines 
32-59, corporation name for example); and 

objects that are within the field of view of the image capture device. 

13. Claims 12-17 and 19 are apparatus claims that correspond to the method claims 
1-6 and 8 respectively and are therefore rejected on the same grounds but directed to 
an apparatus instead of a method. 

14. Claim 25 is similar to claim 1 and is rejected on the same grounds. 

15. As to claim 26, Kogan further teaches annotating information such as landmarks 
to an image ([0012]). 
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16. As to claim 29, Kogan further teaches that the image metadata includes field of 
view metadata ([0015]). 

1 7. As to claim 30, Kogan further teaches using field of view metadata to identify 
objects and events that appear in the image or background of the image ([0016]). 

18. Claim 27 is rejected under 35 U.S.C. 103(a) as being unpatentable over US 
6,469,698 (Fukahori) in view of US 2004/0021780 (Kogan) further in view of US 
5,978,804 (Deitzman). 

1 9. As to claim 27, see the rejection of claim 25 and note that what neither Fukahori 
nor Kogan teach is that flora and fauna that can be found in the location associated with 
the location name is additional inference information. However, Deitzman teaches a 
database for storing flora and fauna associated with specific regions (C30 L61-67). 
Therefore it would have been obvious to one of ordinary skill in the art at the time the 
invention was made to have found flora and fauna data as additional inference data if a 
user is interested in the flora and fauna from where an image was captured as this 
would allow for the information associated with the image to be more customizable for a 
user. 

20. Claim 28 is rejected under 35 U.S.C. 103(a) as being unpatentable over US 
6,469,698 (Fukahori) in view of US 2004/0021780 (Kogan) further in view of US 
6,690,883 (Pelletier). 
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21 . As to claim 28, see the rejection of claim 25 and note that neither Fukahori nor 
Kogan teach that atmospheric associated with the location name is additional inference 
information. However, Pelletier teaches annotating an image with the weather 
conditions at the time of capture (C6 L12-28). Therefore it would have been obvious to 
one of ordinary skill in the art at the time the invention was made to have associated 
atmospheric conditions (weather) for the location associated with the location at the 
time of capture with the image as this would allow for the image to be annotated more 
thoroughly. 

22. Claims 7 and 18 are rejected under 35 U.S.C. 103(a) as being unpatentable over 
US 6,469,698 (Fukahori) in view of US 2004/0021780 (Kogan) further in view of US 
6,961,096 (Tsujimoto). 

23. As to claim 7, see the rejection of claim 1 and note that Fukahori does not teach 
printing the image, the image metadata, and selected inference information. 

However, Tsujimoto teaches printing image data along with associated meta- 
data and other descriptive information, such as a location name (See Fig. 6 and Column 
8 lines 61-67 and Column 1 1 lines 41-48). This feature allows for the benefit to the user 
of being able to identify the circumstances surrounding an image capture including a 
place name (Column 8 lines 47-51 and Column 1 1 lines 41-48). 

Therefore it would have been obvious to one of ordinary skill in the art at the time 
the invention was made to have made prints of the images taken by the method of 
Fukahori and to have further printed meta-data and the associated inference data as is 
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done in a method taught by Tujimoto as discussed above as this would allow a user to 
identify when and where the image was taken. 

24. Claim 18 is an apparatus claim that corresponds to the method claim 7 and is 
therefore rejected on the same grounds but directed to an apparatus instead of a 
method. 

25. Claims 9-11 and 20-22 are rejected under 35 U.S.C. 103(a) as being 
unpatentable over US 6,469,698 (Fukahori) in view of US 2004/0021780 (Kogan) 
further in view of US 2004/01 14042 (Paolini). 

26. As to claim 9, see the rejection of claim 1 and note that what Fukahori does not 
explicitly teach is searching a first database using image metadata to identify inference 
information; and searching a second database using the inference information to 
identify additional inference information. 

However, Paolini searching a first database (a local database) using the image 
metadata to identify the inference information ([0032]); and searching a second 
database (a web-based batabase, for example) using the inference information to 
identify additional inference information ([0034]). 

Therefore it would have been obvious to one of ordinary skill in the art at the time 
the invention was made to have searched a second larger database if the first database 
did not find the information that the user required ([0034] of Paolini). 
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27. As to claim 10, see the rejection of claim 1 and note that what Fukahori does not 
explicitly teach is associating the image metadata with a series of images taken over a 
period of time. 

However, Paolini teaches associating metadata with a series of images taken 
over a period of time ([0019], note the system may be used to annotate videos). 

Therefore it would have been obvious to one of ordinary skill in the art at the time 
the invention was made to have added metadata to a series of images taken over a 
period of time as is done in Paolini to the method of Fukahori as this would allow for 
easier identification of videos as well as still images. 

28. As to claim 11 see the rejection of claim 1 and note that Fukahori does not 
explicitly teach is associating the image metadata with a series of images taken over a 
period of time while the location of the image capture device was changing. 

However, Paolini teaches associating metadata with a series of images taken 
while the location of the image capture device was changing ([0019], the system may be 
used to capture data associated with video, or see Fig. 3 and note that the method of 
Fig. 3 may be carried out repeatedly). 

Therefore it would have been obvious to one of ordinary skill in the art at the time 
the invention was made to have associated image capture metadata with a series of 
images where the location was changing as is done in Paolini in the method of Fukahori 
as this would allow for the location where the series of images were recorded to be 
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determined separately and would help to identify the images for a user. 

29. Claims 20-22 are apparatus claims that correspond to the method claims 7 and 
9-11 respectively and are therefore rejected on the same grounds but directed to an 
apparatus instead of a method. 

30. Claims 23 and 24 are rejected under 35 U.S.C. 103(a) as being unpatentable 
over US 2001/0015759 (Squibbs) in view of US 2001/0041020 (Shaffer) 

31 . As to claim 23, Squibbs teaches a method comprising: 

storing image data representing pixels in a captured image ([0104]); 

storing image metadata representing data associated with conditions at the time 
that the image was captured ([01 1 3]-[01 1 5]); 

generating inference metadata by searching information databases using at least 
a portion of the image metadata ([0049] note that the semantic location data is assigned 
to the image by a user and the information has to come from some sort of database, 
even if that database is only information contained in the memeory of the user); and 

matching the inference metadata with the image data ([0049]). 

What Squibbs does not explicitly teach is calculating a confidence factor relating 
to the matched inference data to rate how closely the inference metadata matches the 
captured image. However, Shaffer teaches using a confidence factor when making 
calculations with subjective data about an image such as the location of the image 
([0021]). Therefore it would have been obvious to one of ordinary skill in the art at the 
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time the invention was made to have used a confidence factor to determine how closely 
the matched inference data of Squibbs, which is subjective data, matches the captured 
image as this would allow for the closest match to be used when collecting inference 
metadata. 

As to claim 24, see the rejection of claim 23 and note that Squibbs further 
teaches the method of claim 23 further comprising: storing an identity of a person 
supervising the match (USER ID, see Fig. 4). 

Conclusion 

Any inquiry concerning this communication or earlier communications from the 
examiner should be directed to Dillon Durnford-Geszvain whose telephone number is 
(571)272-2829. The examiner can normally be reached on Monday through Friday 8 
am to 5 pm. 

If attempts to reach the examiner by telephone are unsuccessful, the examiner's 
supervisor, David Ometz can be reached on (571 ) 272-7593. The fax phone number for 
the organization where this application or proceeding is assigned is 571-273-8300. 
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Information regarding the status of an application may be obtained from the 
Patent Application Information Retrieval (PAIR) system. Status information for 
published applications may be obtained from either Private PAIR or Public PAIR. 
Status information for unpublished applications is available through Private PAIR only. 
For more information about the PAIR system, see http://pair-direct.uspto.gov. Should 
you have questions on access to the Private PAIR system, contact the Electronic 
Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a 
USPTO Customer Service Representative or access to the automated information 
system, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000. 

Dillon Durnford-Geszvain /David L. Ometz/ 

Supervisory Patent Examiner, Art 
Unit 2622 



